Papers with spoken language understanding

44 papers
OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understanding (2023.acl-demo)

Copied to clipboard

Challenge: Spoken Language Understanding (SLU) is a task-oriented dialogue system . open-source toolkit provides a unified, modularized, and extensible toolkit for SLU .
Approach: They introduce an open-source toolkit to provide a unified toolkit for spoken language understanding.
Outcome: The proposed toolkit unifies 10 models for both single-intent and multi-intention scenarios.
Neural Lexicons for Slot Tagging in Spoken Language Understanding (N19-2)

Copied to clipboard

Challenge: lexicons or gazettes are used to improve slot tagging in spoken language understanding systems.
Approach: They develop models that encode lexicon information as neural features for use in a long-short term memory neural network.
Outcome: The proposed models improve slot tagging with lexicons and gazettes . the results could be used to improve other natural language applications .
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention (2020.coling-industry)

Copied to clipboard

Challenge: Existing methods to integrate hypotheses into speech recognition systems are noisy and can cause information loss.
Approach: They propose to integrate hypotheses into multi-task learning and transfer learning to improve performance.
Outcome: The proposed model improves domain and intent classification by 19% and 37% compared to current methods . the proposed model could recover transcription and rewrite the query for a better understanding .
Scaling Multi-Domain Dialogue State Tracking via Query Reformulation (N19-2)

Copied to clipboard

Challenge: Using a pointer-generator network, we model the reference resolution task as a dialogue context-aware user query reformulation task.
Approach: They propose a pointer-generator network and a novel multi-task learning setup to model dialogue state tracking and referring expression resolution tasks using a dialogue context-aware user query reformulation task.
Outcome: The proposed model improves absolute F1 on internal and public benchmarks.
The Alexa Meaning Representation Language (N18-3)

Copied to clipboard

Challenge: a new meaning representation language for spoken language is introduced for Alexa . AMRL provides a common representation for how people communicate in spoken language . there is no mechanism to represent ambiguity, forcing the choice of a fixed interpretation for ambiguous utterances.
Approach: They introduce a meaning representation for spoken language, the Alexa meaning represent language . they use a spoken language dataset to collect a sample of utterances from eight domains .
Outcome: The proposed representation provides a common representation for spoken language understanding . it supports cross-domain queries, fine-grained types, complex utterances and composition . the proposed representation was released to developers at a trade show in 2016 .
Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding (2023.acl-industry)

Copied to clipboard

Challenge: Larger encoders can improve accuracy for spoken language understanding (SLU) but are difficult to use given the inference latency constraints of online systems.
Approach: They propose to use a larger 170M parameter BERT encoder that shares representations across languages, domains and tasks for SLU.
Outcome: The proposed encoders achieve state-of-the-art performance on numerous NLP tasks.
SpeechLLMs for Large-scale Contextualized Zero-shot Slot Filling (2025.emnlp-industry)

Copied to clipboard

Challenge: Slot filling is a key subtask in spoken language understanding (SLU) . recent advent of speech-based large language models has opened new avenues for speech understanding .
Approach: They propose to improve slot-filling task by creating an empirical upper bound for the task . they propose to use a speech-based large language model to integrate speech and text modalities .
Outcome: The proposed model improves slot filling performance while reducing generalization gaps.
MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder (2025.acl-industry)

Copied to clipboard

Challenge: Multilingual automatic speech recognition (ASR) in the medical domain is a critical foundational task, serving a wide range of downstream applications such as speech translation, spoken language understanding, and voice-activated assistants.
Approach: They present the first multilingual medical ASR dataset and the first collection of small-to-large end-to end medical APR models spanning five languages: Vietnamese, English, German, French, and Mandarin Chinese.
Outcome: The proposed model covers Vietnamese, English, German, French, and Mandarin Chinese, and is the first multilingual ASR dataset across five languages.
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings (2022.findings-emnlp)

Copied to clipboard

Challenge: a new study addresses the challenge of learning semantic representations from speech signals . speech-based semantic representation can be used for speech mining and spoken language understanding .
Approach: They propose a multimodal sequential autoencoder that converts speech signals into hidden units . they propose s-HuBERT to induce meaning through knowledge distillation .
Outcome: The proposed model achieves a moderate correlation with human judgments without labels or transcriptions.
A Progressive Model to Enable Continual Learning for Semantic Slot Filling (D19-1)

Copied to clipboard

Challenge: Existing approaches to slot filling training on large scale data are inefficient and require multiple trainings.
Approach: They propose a slot filling model that transfers previously learned knowledge to a small size expanded component and enables it to be fast trained to learn from new data.
Outcome: The proposed model outperforms existing models on two benchmark datasets by 4.24% and 3.03% on the same dataset.
An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking (P18-1)

Copied to clipboard

Challenge: a dialogue state tracker is a core component in most of today's spoken dialogue systems . slot-filling dialogues are composed of a predefined set of slots that need to be filled through the conversation .
Approach: They propose an E2E architecture that extracts unknown slot values while still achieving state-of-the-art accuracy on the standard DSTC2 benchmark.
Outcome: The proposed architecture achieves state-of-the-art accuracy on the DSTC2 benchmark while retaining predefined slot values.
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training (2024.eacl-long)

Copied to clipboard

Challenge: End-to-end (E2E) spoken language understanding models are constrained by the cost of collecting speech-semantics pairs.
Approach: They propose a model that learns E2E SLU without speech-semantics pairs . they propose cross-modal selective self-training (CMSST) to address imbalance and noise issues .
Outcome: The proposed model learns E2E SLU without speech-semantics pairs . the proposed model requires the domains of speech-text and text-sensitization to match .
Distributionally Robust Finetuning BERT for Covariate Drift in Spoken Language Understanding (2022.acl-long)

Copied to clipboard

Challenge: Covariate drift can occur when there is a drift between training and testing regarding what users request or how they request it.
Approach: They propose a method that exploits natural variations in data to create a covariate drift in spoken language understanding datasets.
Outcome: The proposed method improves robustness against covariate drift in spoken language understanding (SLU) it shows that a state-of-the-art model suffers performance loss under this drift.
UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions (2024.naacl-long)

Copied to clipboard

Challenge: Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model’s behavior and surpassing performance of task-specific models.
Approach: They adapt a pre-trained automatic speech recognition model to additional tasks using single-token task specifiers.
Outcome: The proposed model can generalize to new datasets and languages for seen task types.
SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling (2020.emnlp-main)

Copied to clipboard

Challenge: Slot filling and intent detection are two main tasks in spoken language understanding systems.
Approach: They propose a non-autoregressive slot filling model with two-pass iteration mechanism to handle uncoordinated slots problem.
Outcome: The proposed model significantly outperforms previous models in slot filling task while speeding up decoding.
AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing models focus on the single intent scenario, ignoring the fine-grained multiple intents information integration for token-level slot prediction.
Approach: They propose an Adaptive Graph-Interactive Framework for joint multiple intent detection and slot filling . they propose an intent-slot graph interaction layer to model the strong correlation between the slot and intents .
Outcome: The proposed framework improves on three multi-intent datasets and new state-of-the-art performance on single-intention datasets.
Continual Learning Long Short Term Memory (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to prevent catastrophic forgetting in neural networks are based on the stability-plasticity dilemma, but only a limited size of old data is available.
Approach: They propose a Continual Learning Long Short Term Memory cell in Recurrent Neural Network (RNN) that considers the state of each individual task's output gates and the correlation of the states between tasks.
Outcome: The proposed method significantly improves on spoken language understanding tasks over state-of-the-art approaches.
The Spoken Language Understanding MEDIA Benchmark Dataset in the Era of Deep Learning: data updates, training and evaluation tools (2022.lrec-1)

Copied to clipboard

Challenge: a growing number of studies address the spoken language understanding domain through a simple task like speech intent detection.
Approach: They focus on the french MEDIA SLU dataset, which is distributed since 2005 . they propose a recipe for its use, including data preparation, training and evaluation scripts .
Outcome: The MEDIA SLU dataset is used as a benchmark dataset for a large number of research projects.
Enhancing the generalization for Intent Classification and Out-of-Domain Detection in SLU (2021.acl-long)

Copied to clipboard

Challenge: Existing methods for intent classification are expensive to collect and train . evaluators have shown that the ability to detect out-of-domain utterances is limited .
Approach: They propose to train a model with only IND data while supporting both intent classification and OOD detection.
Outcome: The proposed model improves on existing models and strong baselines on four datasets.
Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding (P18-1)

Copied to clipboard

Challenge: Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available.
Approach: They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN.
Outcome: The proposed approach significantly improves learning effectiveness when a small number of training examples are available.
A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding (D19-1)

Copied to clipboard

Challenge: Intent detection and slot filling are two main tasks for building a spoken language understanding system.
Approach: They propose a framework to incorporate intent information into slot filling tasks . they use a joint model with Stack-Propagation to capture intent semantic knowledge .
Outcome: The proposed model outperforms existing models on two publicly available datasets and outperformed existing models by a large margin.
Continual Contrastive Spoken Language Understanding (2024.findings-acl)

Copied to clipboard

Challenge: Recent advances in speech processing require extensive offline training . however, these models struggle to retain their previously acquired knowledge when learning new tasks continuously.
Approach: They propose a method that relies on experience replay and contrastive learning to preserve the learned representations by pulling closer samples from the same class and pushing away the others.
Outcome: The proposed method preserves the learned representations by pulling closer samples from the same class and pushing away the others.
Modeling with Recurrent Neural Networks for Open Vocabulary Slots (C18-1)

Copied to clipboard

Challenge: Existing approaches to filling slots that take on values from a virtually unlimited set have been lacking in the natural language area.
Approach: They propose a new attention-based recurrent neural network (RNN) model that captures the concept: Understanding the role of a word may vary according to how long a reader focuses on a particular part of . sentence.
Outcome: The proposed model outperforms existing models with respect to discovering ‘open-vocabulary’ slots without any external information, such as a named entity database or knowledge base.
Syntactic Graph Convolutional Network for Spoken Language Understanding (2020.coling-main)

Copied to clipboard

Challenge: Existing work on slot filling and intent detection builds joint models without prior knowledge of linguistic knowledge.
Approach: They propose a joint model that integrates syntactic structure for learning slot filling and intent detection jointly.
Outcome: The proposed model outperforms existing models on two public benchmark datasets and further improves on slot filling and intent detection.
Source Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language (C18-1)

Copied to clipboard

Challenge: a study aims to develop a language transferring system to avoid the trouble of acquiring and labeling a new big SLU corpus . general-purpose translators cannot handle the lot of semantic labels, not to mention cultural differences . a RL-based language transfer method can be used to adapt the adapted translator to a target language .
Approach: They propose to use reinforcement learning to adapt a spoken language understanding model to a target language.
Outcome: The proposed language transferring method improves domain classification accuracy by 22% compared with naive translation . the proposed language transfer method can be used on Chinese to English translators with more proper slot tags .
Federated Learning for Spoken Language Understanding (2020.coling-main)

Copied to clipboard

Challenge: Existing methods to improve robustness of models focus on a single dataset . but, there are few studies on how to combine merits of different datasets .
Approach: They propose a federated learning framework that could unify datasets and tasks . they propose MV-Encoder as backbone of the framework to provide multi-granularity text representations .
Outcome: The proposed framework improves on two SLU benchmark datasets and federated learning settings.
Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis (2023.findings-emnlp)

Copied to clipboard

Challenge: Latent Synthesis is an efficient textual data utilization framework for end-to-end speech processing models . labeled speech data are scarcer and more expensive for collection compared to textual ones .
Approach: They propose a textual data utilization framework for E2E speech processing models . they train a latent synthesizer to convert textual information into an intermediate latent representation .
Outcome: The proposed framework improves on low-resource speech recognition and spoken language understanding tasks.
Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding (N19-1)

Copied to clipboard

Challenge: Existing models that use contextual information of dialogues to improve spoken language understanding (SLU) select the wrong history when the histories are similar in content.
Approach: They propose time-aware models that automatically learn the latent time-decay function of the history without a manual time- decay.
Outcome: The proposed models achieve higher F1 scores than state-of-the-art models on a benchmark dataset .
Enhancing Code-Switching for Cross-lingual SLU: A Unified View of Semantic and Grammatical Coherence (2023.emnlp-main)

Copied to clipboard

Challenge: Existing models rely on annotated training data, limiting their scalability to low-resource languages.
Approach: They propose a method termed SoGo for zero-shot cross-lingual SLU that uses keywords as substitution options to extract keywords and a token-level alignment strategy to ensure grammatical coherence.
Outcome: The proposed method improves zero-shot cross-lingual SLU across nine languages on MultiATIS++.
Simulating ASR errors for training SLU systems (L18-1)

Copied to clipboard

Challenge: Existing methods to simulate automatic speech recognition errors from manual transcriptions are not available during training of the SLU model.
Approach: They propose to use acoustic and linguistic word embeddings to define a similarity measure between words to predict ASR confusions.
Outcome: The proposed method significantly improves the performance of spoken language understanding systems.
Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference (P19-1)

Copied to clipboard

Challenge: Existing models for SLU use explicit memory representations, but the context memory is under-exploited.
Approach: They propose a dialogue logistic inference task to consolidate the context memory with SLU in a multi-task framework.
Outcome: The proposed model improves slot filling and domain classification performance in a multi-task framework.
A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling (P19-1)

Copied to clipboard

Challenge: Existing models for slot filling and intent detection lack bi-directional interrelated connections between the intent and slots.
Approach: They propose a bi-directional interrelated model for slot filling and intent detection that uses an SF-ID network to establish direct connections between the two tasks to promote each other mutually.
Outcome: The proposed model improves on ATIS and Snips datasets in sentence-level semantic frame accuracy and improves performance on the two tasks.
Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning (2023.acl-long)

Copied to clipboard

Challenge: Recent work on spoken video grounding challenges extracting semantic information from speech . previous studies focused on textual queries, but recent work focuses on spoken queries .
Approach: They propose a framework for weakly-supervised spoken video grounding to represent cross-modal semantics without expensive temporal annotations.
Outcome: The proposed framework is more efficient than existing methods.
Incorporating Instructional Prompts into a Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling (2022.coling-1)

Copied to clipboard

Challenge: Existing approaches to multiple intent detection and slot filling focus on task-specific components to capture the relationships between intents and slots.
Approach: They propose a Unified Generative framework that captures the relationships between intents and slots in an utterance and formulates the task as a question-answering problem.
Outcome: The proposed framework surpasses baselines on full-data and multi-intent benchmarks on 5-shot and 10-shot scenarios.
Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding (2022.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to translate spoken language understanding into low-resource languages are limited to implicit alignment and disregard the inherent semantic structure in SLU.
Approach: They propose to model utterance-slot-word structure by a multi-level contrastive learning framework . they also propose a label-aware joint model leveraging label semantics to enhance alignment .
Outcome: The proposed model improves performance on two zero-shot cross-lingual datasets.
On the Evaluation of Speech Foundation Models for Spoken Language Understanding (2024.findings-acl)

Copied to clipboard

Challenge: Spoken language understanding evaluation (SLUE) benchmarks are used to benchmark complex spoken language understanding tasks on natural speech.
Approach: They propose a set of benchmark tasks to evaluate spoken language understanding on natural speech . they use pre-trained speech foundation models to evaluate the utility of different SFMs .
Outcome: The proposed framework outperforms pre-trained speech foundation models on natural speech . the proposed framework also outperformed self-supervised SFMs on the sequence generation tasks .
MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)

Copied to clipboard

Challenge: Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems.
Approach: They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts .
Outcome: The proposed framework achieves state-of-the-art on three benchmark SLU datasets.
ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated impressive capabilities in language generation and general task performance, but their application to spoken language understanding remains challenging.
Approach: They propose an Entity-level Language Model framework which reformulates slot-filling as an entity recognition task and introduces a new concept, Chain of Intent, to enable step-by-step multi-intent recognition.
Outcome: The proposed framework outperforms strong baselines such as Uni-MIS and achieves gains of 3.7% and 3.1% on MixATIS and MixSNIPS.
Synergistic Augmentation: Enhancing Cross-Domain Zero-Shot Slot Filling with Small Model-Assisted Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Existing approaches to slot filling are limited due to data scarcity and timeconsuming efforts.
Approach: They propose a framework that harnesses the power of a small model to augment inferential capabilities of LLMs without additional training.
Outcome: The proposed framework improves slot filling performance on a spoken language dataset and a NER dataset.
Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods to increase the robustness of pre-trained language models (PLMs) against unseen ASR systems produce noisy inputs for SLU models, which can significantly degrade their performance.
Approach: They propose to introduce ASR-plausible noises into pre-trained language models by cutting off the non-causal effect of noises.
Outcome: The proposed method improves the robustness and generalizability of SLU models against unseen ASR systems by cutting off the non-causal effect of noises.
Speech Discrete Tokens or Continuous Features? A Comparative Analysis for Spoken Language Understanding in SpeechLLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Speech Large Language Models (SpeechLLMs) have emerged as dominant speech processing approaches.
Approach: They compare self-supervised learning-based discrete and continuous features . they compare performance across six spoken language understanding-related tasks .
Outcome: The proposed models outperform discrete tokens and continuous features in six spoken language understanding-related tasks.
Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants (2024.lrec-main)

Copied to clipboard

Challenge: Recent studies show voice assistants do not perform equally well for everyone . however, research on demographic robustness of speech technologies is still scarce .
Approach: They propose a statistical method to detect demographic bias using a large dataset with controlled demographic tags.
Outcome: The proposed method shows statistically significant differences in performance across age, dialectal region and ethnicity.
Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks (2025.acl-long)

Copied to clipboard

Challenge: a growing number of recorded human speech is recorded for automated processing, resulting in errors in the transcripts . a configurable framework is proposed to analyze transcript noise impact across noise levels and transcript-cleaning techniques.
Approach: They propose a configurable framework for assessing task models in diverse noisy settings . framework facilitates investigation of task model behavior, which can support effective SLU solutions.
Outcome: The proposed framework can analyze model behavior in various noise levels and transcript-cleaning techniques.
What Has LeBenchmark Learnt about French Syntax? (2024.lrec-main)

Copied to clipboard

Challenge: Pretrained acoustic models are increasingly used for downstream speech tasks such as automatic speech recognition, speech translation, spoken language understanding or speech parsing.
Approach: They propose to probing a pretrained acoustic model for French for syntactic information using the Orféo treebank.
Outcome: The proposed model is trained on 7k hours of spoken French and obtained reasonable results on tasks that require higher level linguistic knowledge.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations